Skip to content

Confidence guard: per-projection standard error, not historical RMSE - #19

Merged
zebraengine merged 1 commit into
mainfrom
fix/projection-uncertainty
Aug 24, 2026
Merged

Confidence guard: per-projection standard error, not historical RMSE#19
zebraengine merged 1 commit into
mainfrom
fix/projection-uncertainty

Conversation

@zebraengine

Copy link
Copy Markdown
Owner

Problem

Issue #4: the amp controller's confidence guard treated a will_trip: false verdict as untrustworthy when the projected plateau was within k × fit_rmse_c of the trip point. Right idea, wrong denominator — fit_rmse_c describes how well the exponential model fit past sessions' ramps, a single constant, while the uncertainty of this projection, right now varies enormously over a trajectory window's life. One number for both cases made the guard too loose early (when it matters most) and too tight late (costing charge rate for nothing). The owner's charger demonstrated the late case four times on Aug 21–22: caps at 0.3–0.6 °C margins the old guard scored at 1.0–1.8 σ.

Code touched

wallmonitor/thermal.py

  • _project_t_inf() returns (t_inf, se). T_inf is the intercept of the existing OLS fit on x = exp(−t/τ), so its standard error is the closed-form intercept SE: s·sqrt(1/n + mean_x²/Sxx) with s² = SSE/(n−2) — eight lines inside a function that already had every input (as the issue predicted). se is None for degenerate windows (n ≤ 2, or the flat-decay fallback); a noiseless flat window regresses to exactly 0.0, which is numerically true and physically overconfident, so —
  • predict() publishes steady_state_se_c on trajectory-basis forecasts, floored at 0.1 °C (the handle sensor's quantization step). Model/hypothetical bases don't get one: their uncertainty is dominated by the ambient estimate, not a regression, and pretending otherwise would be a different lie. The value rides into forecast_samples via the raw payload; no schema change.
  • The second _project_t_inf caller (_recent_steady_ambient) ignores the SE explicitly.

contrib/derate_amp_control.py

  • The guard divides the margin by steady_state_se_c when the payload carries one, falling back to fit_rmse_c (older server, missing SE) — so a daemon updated ahead of its server keeps exactly the old behavior. Reason strings name which was used (… σ vs proj se / … σ vs fit rmse), so future field reports are unambiguous.
  • --forecast-confidence-k default stays 2.0: against a real standard error, 2σ is a meaningful ~95 % statement rather than an arbitrary multiplier. Not silently retuned.

Docs: the guard paragraph in amp-control.md rewritten around the per-projection error with the measured numbers; one sentence in thermal-model.md's live-forecast section.

Risk

  • The guard becomes less conservative near the plateau — deliberately. Replayed on a real 43 A stretch: at a 1.1 °C margin the old guard read 3.5 σ, the new one 11 σ. The Aug 21–22 production caps (0.3–0.6 °C margins) would mostly not fire under the new guard when the window is mature (SE at floor → 3–6 σ). That is the issue's stated intent — proximity plus a trustworthy forecast is not the danger — but it is a behavior change on a safety-adjacent path; the cap path for will_trip: true and every other guard (confirm ticks, restore backoff, floors) is untouched, and alert 40 itself remains the hardware's own backstop.
  • Early-window behavior tightens, the direction that matters: a 24-second-old projection now carries SE ≈ 0.8 °C, so the guard distrusts exactly the ticks the constant used to bless.
  • The 0.1 °C floor is a judgment call: below the sensor's LSB, "confidence" is an artifact. It also bounds worst-case guard sensitivity (k = 2 → margins under 0.2 °C always distrusted).
  • Kalman filter deliberately not built, per the issue: the OLS SE captures most of the benefit for a fraction of the blast radius.

Verification

  • 101/101 tests. New: _project_t_inf SE is >3× wider on the first quarter of a synthetic ramp than on a window that has seen the bend, exact-flat behavior documented; guard prefers the SE (wide SE + small nominal RMSE → caps, where the old guard slept), trusts a tight projection near the trip (steps up where the old guard stepped down), and falls back to fit_rmse_c byte-for-byte when the SE is absent.
  • Replayed over a real 27-minute 43 A charging stretch from the owner's database copy (table in the commit): raw SE 0.81 → 0.11 → 0.04 °C as the window matures; sigma-vs-SE and sigma-vs-RMSE diverge exactly as the issue predicted at both ends.
  • Backtest harness (contrib/backtest_derate_amp_control.py) runs predict()+decide() unchanged and now exercises the new path for free; worth a run against the production DB after a few real sessions to observe cap frequency before/after.

Deploy: server (git pull + restart for the poller's forecasts) and the amp-control daemon pick this up independently; either order is safe thanks to the fallback.

Closes #4.

🤖 Generated with Claude Code

…storical RMSE

The amp controller distrusted a 'no trip' verdict when the projected
plateau sat within k x fit_rmse_c of the trip point — but fit_rmse_c is
a model-adequacy constant across past sessions, while projection
uncertainty varies enormously over a trajectory window's life. The
regression already had the real thing in closed form: _project_t_inf now
returns the OLS intercept standard error alongside the plateau, predict()
publishes it as steady_state_se_c (floored at the sensor's 0.1 C step so
a noiseless window can't claim impossible confidence), and the guard
divides the margin by it, falling back to fit_rmse_c when absent.

Measured on a real 43 A stretch: raw SE 0.81 C at 24 s into the window,
0.11 C by three minutes, ~0.04 C near the plateau — so the guard now
distrusts exactly the early ticks the old constant scored as confident,
and trusts mature projections it used to step down on.

Closes #4.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@zebraengine
zebraengine merged commit 6f277cd into main Aug 24, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Confidence guard uses fit_rmse_c as a proxy for projection uncertainty — the regression already computes the real thing

1 participant